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ABSTRACT 

The purpose of this study is to explore the potential impact of support materials on 
student spreadsheet skill acquisition. Specifically, this study examines the use of an online 
spreadsheet simulation tool versus a printed book across two independent student groups. This 
study hypothesizes that the online spreadsheet simulation tool will have a greater positive impact 
on student performance than a printed book. Student performance was measured using scores on 
10 individual spreadsheet projects and an overall average of these 10 scores. T-tests were used 
to determine significance since the project scores are independent and means were compared 
only across two groups. Significant differences were observed between the groups for two of the 
ten projects, but further research is needed to identify additional factors that may influence these 
findings. 
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INTRODUCTION 

University professors teaching survey of management information system courses 
routinely strive to find methods of engaging their students (Mukherjee, 2000; Ramiller, 2003). 
Spreadsheets used for business modelling are relevant to diverse disciplines like marketing, 
finance and accounting, and engaging to students (Alexander, 1996; Ganesh & Paswan, 2010; 
Van Mourik, 2006). Courses in these disciplines are diverse in terms of the appropriate mix of 
conceptual material and activities designed for enhanced skill proficiencies. There is also 
diversity in the types of students enrolling in such courses (Sabry & Barker, 2009). Accreditation 
plays a central role in any academic institution and schools of business are no exception placing 
great importance on inclusion into the Association to Advance Collegiate Schools of Business 
(AACSB). Business schools with AACSB accreditation require programs to provide “learning 
experiences addressing both general and management-specific learning goals” (AACSB, 2006) 
which includes the use of information technology. Using data collected over the last three years, 
two distinct methods for supporting undergraduate student spreadsheet skills acquisition have 
been employed. One method fits the more traditional approach of supplementing instructional 
lectures, both face-to-face and pre-recorded video formats, with a comprehensive spreadsheet 
reference book. The more recent adaptation involves using an online simulation software 
package that employs a methodological approach of “teach me, show me, let me try” for skill 
building (McGraw-Hill, 2005) in an interactive, spreadsheet-like environment. This paper 
investigates differences between the two approaches with regard to their impact on student 
performance. Because of the relatively short period of time that Internet-based, simulation tools 
have been available for student use, a review of the origins of tutorials is included. 

UITERATURE REVIEW 

The introduction of new technology into the education realm will inevitably create debate 
as to its effectiveness. Some educators will argue that technology enhances learning while others 
will take the stance that it may conflict with students’ thought processes (Jackson, Gaudet, 
McDaniel, & Brammer, 2009; Roman, 2009). When printed books were first introduced in large 
numbers in institutions, educators most likely perceived the new tool as either an aid to learning 
or felt threatened as the traditional conveyor of knowledge and certainly often hold different 
views than the textbook on core principles (Cuban, 2007; Leonidou, Kaminarides, & Panayides, 
2007). When electronic calculators became widely available the debate as to whether students 
should be allowed to use them in class was almost immediate (Fleener, 1995; Herman, 2007). 
Personal computers added to this discussion, and as the costs of both computers and productivity 
software continues to decline additional fuel is being added to fire. 

Electronic spreadsheet software, like Lotus 1-2-3® and Microsoft Excel® are useful to 
students in various ways. First, more disciplines are beginning to integrate computer-based skills 
into their curriculum (Alexander, 1996; Ganesh & Paswan, 2010; McKeown & Piercy, 1997). 
Perhaps more importantly, spreadsheet software is very useful for modelling business problems 
and enabling data analysis using what-if, pivot tables, and even statistical analysis functionality 
(Van Mourik, 2006). 

The transference of knowledge in the use of spreadsheet software includes demonstrating 
its various uses and then providing the student with sufficient opportunities as to reasonably 
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ensure that competencies are acquired. One method may that produces results is the 
demonstration and practice method. Instructors in this approach demonstrate to students how 
selected functions work and then ask students to work through problems that require the 
demonstrated skills application. Often these results are limited by the student becoming 
dependent on the professor to demonstrate, observe, practice, and then examine (Vitt, 2006). 
Howard (2007), suggests that “we must look forward...testing promising developments...while 
holding on to the established practices of the past” (p.31). Moving forward may include the use 
of supplemental reference materials, like video tutorials, online guides, and software simulations. 
Tutorials can be enhanced by using a book that serves as a reference guide and can engage 
students actively and more deeply in the learning process (Gareis, 2007). 

The progression from tutorial-type books to an electronic tutorial or simulation may 
allow students to test their abilities within a “realistic, real-time environment” (Newman, 2007, 
p. 466). Natarajan (2006) suggests that the Internet allows students to have more opportunities 
and overcomes the challenges of time and travel restraints of face-to-face education. When 
students are conducting simulation activities, they are also learning about processes. According 
to Foti (2007), simulations are tools used for learning rather than tools for productivity. In this 
instance, productivity would be constrained to work-place environment effectiveness. Batscha & 
Moloney (2005) posits that simulations when used as a study tool, are helpful for preparing for 
and passing examinations; Polito, Kros, and Watson (2004) had similar results when examining 
simulation-based tools for improving exam scores, and other studies suggests that simulations 
are beneficial for critical thinking and problem-solving skills in clinical settings (Feingold, 
Calaluce, & Kallen, 2004). 

In the business environment, needs are met using simulation modelling and analysis 
(Hwarng, 2001). Today’s business environment presents a series of complex tasks; tasks that 
students will face as they embark in a competitive and often unforgiving workplace. “We must 
equip our business students with simulation modelling skills or leave them incompetent for the 
dynamic business world of today” (p. 74). Tools that are prevalent in organizations and the skills 
to utilize them are necessities for today’s college students. Simulation tools are becoming 
increasingly available to people and the need to understand the impact of such tools is critical. 
Bert (1998), a civil engineer, proposed that simulation will reassure them [the students] that the 
theory does match the practice. Providing sufficient relevance to enable the learner to utilize an 
spreadsheet-based simulation to model business problems and then to practically apply those 
using spreadsheet skills is the ultimate goal. 

METHODOLOGY 

It is hypothesized that there are significant positive differences between college students’ 
individual spreadsheet project scores when a computer-based simulation spreadsheet learning 
tool is used as compared to a reference book. Further, there are significant positive differences 
between college students’ spreadsheet composite scores when a computer-based simulation 
spreadsheet learning tool is used as compared to a reference book. 

The initial hypothesis links each individual simulation lesson with the associated 
spreadsheet project score. The second hypothesizes that student’s overall spreadsheet average 
scores among ten projects will indicate a positive significance when the simulation software 
lessons are used as a teaching tool. 
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Participants 

Participants selected in this study were students in established sections of principles of 
management information systems survey courses at one south central, public university. The 
total number of students included in this study is 879; with 730 students using the reference book 
between fall 2007 and summer 2009, and 149 students using the electronic simulation beginning 
in the fall of 2009.. The sections represent a convenience sample and there are no perceived 
differences between these sections in terms of student selection beyond the typical registration 
process. Students that dropped the class or students that completed less than eight out of ten 
projects were not used in our study. 

Data Collection and Procedures 

The data was collected across several semesters beginning in 2007 using spreadsheet 
projects offered through Casegrader for Excel 2007® by Crews and Murphy and published by 
Cengage Learning. The projects in this particular text are scored using an automated engine that 
has been shown to be accurate, consistent, and reliable. According to one author, Dr. Thad Crews 
(personal interview, March 2, 2010), over one million projects have been consistently and 
successfully graded for thousands of students across campuses throughout the U.S. and Canada. 
The Excel Reference text, New Perspectives - Excel 2007 Comprehensive, published by Course 
Technology, was the book used from fall 2007 until the fall of 2009. The book was replaced with 
an Internet-based, simulation package beginning in the fall of 2009. The simulation software in 
use, SimNet® by McGraw-Hill, uses a sophisticated Flash simulation model to emulate Excel 
2007 and provides for the creation of lessons, examinations and other learning support 
mechanisms. The use of two disparate tools, Casegrader for Excel 2007 and SimNet, developed 
separately, is expected to reduce research bias. 

Biographical data was requested and obtained from university records and merged using 
Excel with the student score data. This data included the students’ sex, course section number, 
student identification number, as well as other biographical data. The course section number is 
coded so that face-to-face and online sections are distinguishable, and a simple code to identify 
each type of class offering was inserted for each participant to enable further data analysis for 
future studies. The demographic data are summarized in Table 1 (Appendix). 

Data Analysis 

The students were divided into two groups, those who used the reference book and those 
who used the electronic simulation tool. To test hypothesis HI, a single-tailed t-test was 
performed to compare students’ individual Casegrader project scores between the two groups. 

To test hypothesis H2, a second single-tailed t-test was then performed to compare students’ 
overall project score averages across the two groups. According to Bekiroglu (2001) and Hair et 
al. (1998), when two means taken from independent groups are compared, the t test is sufficient 
for determining if the means are significantly different. In this instance, the mean score of each 
independent project was compared between the group that used a reference book and the group 
that used the software. 
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FINDINGS 

The results of our data analysis were mixed. The hypothesis that states that there are 
significant positive differences between college student’s individual spreadsheet scores when a 
computer-based simulation spreadsheet learning tool is supported in two projects. The results for 
projects 7 and 8 indicated significant support for this hypothesis. The secondary hypothesis 
states that there are significant positive differences between college student’s spreadsheet 
composite scores when a computer-based simulation spreadsheet learning tool is used, was not 
supported. The results of the hypotheses testing are presented in Table 2 (Appendix). Although 
the differences were insignificant in most projects, it should be noted that a positive and growing 
gap was observed between SimNet and the book, especially across projects 5 through 8 as shown 
in Figure 1 (Appendix). In addition, though not significant, the composite score for students 
using SimNet was also greater than the score of those using the book. 

DISCUSSION OF FINDINGS 

The original hypothesis that significant differences would exist between the use of a 
reference book and an online simulation tool is largely a perceived one. As noted, significant 
differences were not observed for several projects. One possible explanation for these results 
may be the specific book used as a reference. The book was written in conjunction with 
Casegrader; thus it provides specific examples that coincide directly with the steps of each 
project. This direct linkage to Casegrader may have led to somewhat inflated scores, as students 
could directly imitate the steps provided in the book. In addition, the Excel book contains 
definitions of terms as well as explanations of how the terms are used within the spreadsheet. 

The SimNet simulation did not precisely follow the projects; and in some instances, key 
spreadsheet competencies were absent. While omission of the book-similar steps was explained 
to the students during orientation and again during lab sessions, many students later commented 
that they did not take additional time to perform their own research to resolve the more arduous 
steps. 

Batscha and Moloney (2005) found that simulations can aid exam success when used as a 
study tool. In the current study, it is likely that most students used SimNet as a help resource 
rather than a study tool. In other words, they accessed SimNet only for assistance on difficult 
steps in each project, as opposed to completing the entire simulation. An increased significance 
for student performance may have resulted had students completed each simulation prior to 
beginning the project. 

Projects 7 and 8 did indicate significance which may be explained by the introduction of 
skills that students are less likely to have encountered previously, such as nested if statements, 
lookup functions, macros, and even database commands. Projects 6 and 10 did not indicate 
significance which was surprising due the complexity of multiple workbooks in project 6 and the 
introduction of data tables and what-if tools in project 10. Project 9 integrates relatively simple 
skills into several worksheets that solve accounting and finance problems. It is posited that 
business students are better equipped to relate the application of the skills in this instance with 
the problems presented for purely contextual reasons. McKeown and Piercy (1997) demonstrated 
this by integrating spreadsheets into introductory marketing and finance courses that resulted in 
students performing significantly higher using a pre- and post-test methodology. Finally, it 
should be noted that the individual learning styles of each student were not taken into 
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consideration. In order to further explain the scores of each project, the student’s individual 
learning style should be taken into account (Baldwin & Sabry, 2003). 

It is interesting to note that, although the differences in scores across the two groups were 
insignificant for most projects, Figure 1 (Appendix) demonstrates a gap in the scores that tends 
to widen in favor of SimNet across the more complicated projects. This is perhaps due in part to 
an increase in faculty-student interaction as the semester progresses. Very little time is spent 
during labs covering details for the first few projects; however, as the projects progress in 
number and rigor, each faculty member describes the project in greater detail, and demonstrates 
examples during the open lab sessions allowing students to ask questions. These observations 
are in line with the findings of Vitt (2006), who indicated that, traditionally, technology skills 
have been taught through active student-instructor interaction, including demonstration, practice, 
and observation. It is likely that these interactions have not yet been replaced by SimNet, but 
rather have been supplemented, especially in the face-to-face classes. Thus, the impact of 
SimNet on student performance may be diluted due to the continued presence of student-teacher 
interaction. 

Additional Research 

Additionally, the computer information systems course in which these assignments are 
assigned is offered in both face-to-face and online venues. Additional analysis is necessary to 
further identify differences between student performance comparing the use of the reference 
book and SimNet within each type of offering, differences in performance using gender as the 
control variable, and differences between majors within the various disciplines represented. 

While each faculty member uses the same course resources, additional studies should be 
considered that compares perceived pedagogical differences between instructors to explore 
implications on student performance. The types of differences proposed should focus on the 
strategic approach to teaching and learning. For example, Hornby, Jennings, & Nulty (2009) 
report that faculty should consider using an action research famework to engage in deep 
approaches to learning, while Mathews (2008) posits that Case-Based Reasoning [CBR] allows 
the student to explore the depth of the material and they actually enjoyed this approach better 
than non case-based approaches to learning. Additionally, Mathews said that it did not matter 
whether the class was internal [face-to-face] or distance-based; and the student’s perception was 
that “CBR enhanced their learning” (p. 26). Specifically, further work should be performed that 
would compare teaching style differences, professor reputation, and student project lab 
participation. 
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APPENDIX 

Table 1: Demographic Data 

Book 

Users 

SimNet 

Users 

Gender 

Male 

440 

85 


Female 

289 

64 

Age 

Average 

24 

23 


Minimum 

19 

19 


Maximum 

55 

41 

Class Type 

Face to Face 

336 

79 


Online 

393 

70 

Program of Study 

Accounting 

98 

20 


Management 

187 

35 


Marketing 

134 

37 


Finance 

74 

17 


Business & Marketing Education 

7 

0 


Economics 

22 

9 


Computer Information Systems 

20 

3 


Business Administration 

5 

1 


Undeclared 

33 

8 


Outside the College of Business 

149 

19 
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Table 2: Data Analysis Results 


Excel Project 

Resource 

Mean 

Standard 

Deviation 

t-Statistic for 
Differences 

Project 1 

Web-Based 

93.40 

22.25 

0.865 

Book 

95.07 

16.73 

Project 2 

Web-Based 

91.31 

15.49 

-0.211 

Book 

91.01 

15.42 

Project 3 

Web-Based 

91.14 

18.68 

0.474 

Book 

91.92 

15.44 

Project 4 

Web-Based 

94.78 

14.62 

-0.396 

Book 

94.28 

12.24 

Project 5 

Web-Based 

84.55 

23.77 

-0.971 

Book 

82.44 

25.80 

Project 6 

Web-Based 

85.17 

26.48 

-1.159 

Book 

82.39 

27.77 

Project 7 

Web-Based 

88.83 

23.68 

-2.516* 

Book 

83.24 

29.14 

Project 8 

Web-Based 

91.33 

13.16 

-3.985** 

Book 

85.96 

21.81 

Project 9 

Web-Based 

86.44 

25.15 

-1.640 

Book 

82.62 

29.27 

Project 10 

Web-Based 

77.87 

28.39 

0.775 

Book 

79.81 

25.06 

Overall 
Project Grade 

Web-Based 

88.48 

11.01 

-1.609 

Book 

86.87 

11.62 

Web-Based N = 
p<.05*, p<.01**. 

[49, Book N = 729. 


Figure 1: Comparison of Excel Project Grades 


Comparison of Excel Case Grades 
(Web-Based Tutorial vs. Book) 
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